Papers with open-ended question answering
Search Augmented Instruction Learning (2023.findings-emnlp)
Copied to clipboard
Hongyin Luo, Tianhua Zhang, Yung-Sung Chuang, Yuan Gong, Yoon Kim, Xixin Wu, Helen Meng, James Glass
| Challenge: | Large language models (LLMs) have been significantly improved by instruction fine-tuning, but still lack transparency and the ability to utilize up-to-date knowledge and information. |
| Approach: | They propose a search-augmented instruction learning model which grounds the language generation and instruction following abilities on complex search results generated by in-house and external search engines. |
| Outcome: | The proposed model outperforms plain LLMs on zero-shot language tasks and can generate both natural and programming languages following natural language guidance and requests. |
Evaluating the Elementary Multilingual Capabilities of Large Language Models with MultiQ (2024.findings-acl)
Copied to clipboard
| Challenge: | a global majority of non-English speakers are underrepresented by large language models . however, most open LLMs are limited in their language coverage . |
| Approach: | They propose a silver standard benchmark for basic open-ended question answering with 27.4k test questions across a typologically diverse set of 137 languages. |
| Outcome: | The proposed model can answer questions in 27.4k questions across 137 languages. |
Where am I? Large Language Models Wandering between Semantics and Structures in Long Contexts (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing evaluations of the open-domain question answering task focus solely on whether the model provides the correct answer. |
| Approach: | They propose to examine the phenomenon of discrepancies in abilities across two distinct tasks—QA and evidence selection—when performed simultaneously. |
| Outcome: | The proposed framework and resources examines the ability of large language models to perform two distinct tasks simultaneously, from the perspective of task alignment. |
Downstream Trade-offs of a Family of Text Watermarks (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) can generate humanlike responses to a variety of requests like writing emails, translating or summarizing content. |
| Approach: | They evaluate the performance of large language models (LLMs) watermarked using three different strategies over a diverse suite of tasks including those cast as k-class classification (CLS), multiple choice question answering (MCQ), short-form generation (e.g., open-ended question answering) and long-form generator (eg. translation) |
| Outcome: | The proposed models can cause significant drops in their effectiveness across a variety of tasks including CLS, MCQ, short-form generation and translation tasks. |
Extrinsic Evaluation of Cultural Competence in Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Prior studies have evaluated models’ knowledge of cultural norms, values, and artefacts, without considering how this knowledge manifests in downstream applications. |
| Approach: | They propose to evaluate model outputs when an explicit cue of culture, specifically nationality, is perturbed in prompts. |
| Outcome: | The outputs of two open-ended question answering and story generation tasks are evaluated when an explicit cue of culture, specifically nationality, is perturbed in the prompts. |
BiMediX: Bilingual Medical Mixture of Experts LLM (2024.findings-emnlp)
Copied to clipboard
Sara Pieri, Sahal Shaji Mullappilly, Fahad Khan, Rao Anwer, Salman Khan, Timothy Baldwin, Hisham Cholakkal
| Challenge: | a new bilingual medical mixture of experts LLM is designed for seamless interaction in both English and Arabic. |
| Approach: | They propose a semi-automated English-to-Arabic translation pipeline with human refinement to ensure high-quality translations. |
| Outcome: | The proposed model outperforms state-of-the-art medical LLMs in Arabic and Arabic . it outperformed the generic Arabic-English bilingual LLM, Jais-30B by 10% and 15% . |
Time-MQA: Time Series Multi-Task Question Answering with Context Enhancement (2025.acl-long)
Copied to clipboard
Yaxuan Kong, Yiyuan Yang, Yoontae Hwang, Wenjie Du, Stefan Zohren, Zhangyang Wang, Ming Jin, Qingsong Wen
| Challenge: | Existing time series models focus on a narrow spectrum of tasks, such as forecasting or anomaly detection. |
| Approach: | They propose a framework that enables natural language queries across multiple time series tasks such as numerical analytical tasks and open-ended question answering with reasoning. |
| Outcome: | The proposed framework enables natural language queries across multiple time series tasks and allows for more advanced and intuitive interactions with temporal data. |